Papers with energy consumption
Evaluating Cost-Efficiency of LLMs in a RAG Setup on Polish Wikipedia: Quality vs. Energy Consumption (2026.eacl-srw)
Copied to clipboard
| Challenge: | Retrieval-augmented generation systems are a dominant paradigm for knowledge-intensive applications. |
| Approach: | They evaluate language models from 4B to 70B parameters within a Polish Wikipedia-based RAG pipeline. |
| Outcome: | The proposed model selection process reduces energy consumption by 83% and improves quality. |
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Quantization-aware training (QAT) is a promising method to lower the implementation cost and energy consumption. |
| Approach: | They propose a method for fast converging QAT of pre-trained Transformers using a layer-wise signal propagation method with the intact signal from the teacher. |
| Outcome: | The proposed method achieves superior accuracy with significantly lower fine-tuning iterations on well-known Transformers of natural language processing as well as computer vision compared to the state-of-the-art methods. |
Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies show that the energy requirements of current NLP models are growing at a rapid, unsustainable pace. |
| Approach: | They investigate ways to measure energy usage and different hardware settings that can be tuned to reduce energy consumption for training and inference for language models. |
| Outcome: | The proposed techniques can reduce energy consumption for training and inference for language models. |
IrEne: Interpretable Energy Prediction for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. |
| Approach: | They propose an interpretable and extensible energy prediction system that predicts inference energy consumption of Transformer-based NLP models. |
| Outcome: | The proposed system predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. |
ColBERT-XM: A Modular Multi-Vector Representation Model for Zero-Shot Multilingual Information Retrieval (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve retrieval effectiveness in high-resource languages are limited due to the lack of high-quality labeled data in non-English languages. |
| Approach: | They propose a modular dense retrieval model that learns from the rich data of a single high-resource language and effectively zero-shot transfers to a wide array of languages. |
| Outcome: | The proposed model performs well against state-of-the-art multilingual retrieval models trained on more extensive datasets in various languages. |
CICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification (2024.findings-acl)
Copied to clipboard
| Challenge: | Contaminated or adulterated food poses a substantial risk to human health. |
| Approach: | They present a dataset of 7,546 text messages describing public food recalls. |
| Outcome: | The proposed model outperforms RoBERTa and XLM-R on classes with low support while reducing energy consumption. |
Bridging Fairness and Environmental Sustainability in Natural Language Processing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a lack of research on the interplay between fairness and environmental impact is a problem in natural language processing . fairness is prone to encode and amplify stereotypical social biases, according to several studies . |
| Approach: | They evaluate a technique to reduce energy consumption of English NLP models by knowledge distillation for its impact on fairness. |
| Outcome: | The proposed method reduces energy consumption and environmental impact of English NLP models. |
MMDEND: Dendrite-Inspired Multi-Branch Multi-Compartment Parallel Spiking Neuron for Sequence Modeling (2025.acl-long)
Copied to clipboard
Kexin Wang, Yuhong Chou, Di Shang, Shijie Mei, Jiahong Zhang, Yanbin Huang, Man Yao, Bo Xu, Guoqi Li
| Challenge: | Vanilla spiking neurons are simplified from complex biological neurons with dendrites, soma, and synapses into single somatic compartments. |
| Approach: | They propose a multi-branch, multi-compartment parallel spiking dendritic neuron with a proportion-adjustable multi-branched structure that enables long-term temporal dependencies. |
| Outcome: | The proposed model achieves better long-sequence modeling capability with fewer parameters and lower energy consumption. |